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(57) Simple calibration of a camera mounted on a 
mobile object is implemented. With a joint means, a tar- 
get apparatus for calibration is physically fixed to the 



mobile object having a camera mounted thereon. The 
positional relation between the vehicle and the target 
apparatus is finely adjusted while visually confirming a 
target point on the vehicle through a scope means. 
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Descripti n 

BACKGROUND OF THE INVENTION 



captured image on a monitor mounted ms.de the vehicle surrounding monitoring system 

[0003] Fig. 23 is a block diagram J" °^mZ ^velg section 1202 recedes the respective 

(Japanese Laid-Open Publication No. 3-99952). In Fig. 23, a 9 svnthe sized image as viewed from a 

mages from a plurality of cameras 1201 mounted on the veh dtaSSSiertad image on a 

virtual view point by perspective transformation. An .mage d ^^^^Z.mmi at the upper 
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1204, resulting in improved safety of the driving. 
(Problems) 
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[0004] The aforementioned monitoring system joins a XTTJ^. 
Ld any displacement at each calibration, 
orientation and position of each came a a ^^ f ^Z^ eacn havin g a known coordinate position in a 
[0005] For example, in a known cal.brat.on method, £^£P»rts eacnn g ^ ^ 

look)" (Shin-gijyutsu Communications, pp. 37-53, June isaoj, oescr y calibration . technology of calibrating a 

[0006] For example, the following techno.ogies are known ^^^^^ £ Opposition No. 
La. sensor system of a transfer line by means ° f ^^ 

7-90494);andtechno.ogyofobta,n,ngan,ns^ No 7 . 27408) However , no 

system in a robot handling apparatus (disclosed n P "^°" ° ^ J mobile objec ts such as vehicles, 

effective calibration technology has ^•* rt ^^_ C 3^^^ such as those for vehicle moni- 
[0007] Moreover, in order to calibrate the This requirement may be 

taring, feature points must be widely d.stnbute d throu J°^j^ m ^ S H ~ £„ is 9 not preferable since a large 
satisfied by installation of huge equipment hav.ng such feature J * location, a vehicle must 
site and extremely high costs are required. *"™^££Z£ ^nf.^enS Since it is expected 
be moved all the way to the location for ^£S^*pment of a simple camera «». 

rn 8 ::^^ 

S displacement of the camera due to a traffic accident or veh.cle runn.ng. 
SUMMARY OF THE INVENTION 

[0008] It is an object of the present invention to enable simp.e high-precision calibration of a camera mounted on a 

mobile object. _„«„„ tn nrp«sent invention includes : a mobile object having a 

[0009] More specifically, a calibration system accord.ng btepmrt invent .on ^ 
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of the mobile object and the target apparatus for positioning the target apparatus with reference to a target point located 

rJoTa] *Mm calibration system of the present invention, the target apparatus has a feature 

light under external control, and the mobile object includes a control means for controlling the hght em.ss.on of the 

55 aiglet P S i—, a calibration system inciudes: a mobile object having a 
hereon- a Sget apparatus for calibrating the camera; and a positional-relation estimafng means ported to at least 
one of the Se object ant the target apparatus for obtaining a positional relation between the mob.le object and the 

Sff^rting to the present Invention, the positional relation between the nwbileobjert and the target 

estimating means. As a result, the mobile object need no longer be accuretely 
£m calibration. Accordingly, calibration of the camera mounted on the mob.le object .s s mph fled 
mm In the calibration system of the present invention, the positional-relation est.mat.ng means has a target data 
King meanTpSided to one of the mobile object and the target apparatus for obtaining a coordinate value of a 
taraet point located on the other in a coordinate system in the one with reference to the target point. 
Hm A tarSl 'apparatus for calibrating a camera mounted on a mobile object according to the present mven ,on 
Kudos iSSSH means for fixing actional relation between the mobile object and the target apparatus to a 

Solely ^apparatus for calibrating a camera mounted on a mobile object according to the .present ^nvention 
Includes a positional-relation estimating means for obtaining a positional relation between the mob.le object and the 

loSrAShod for calibrating a camera mounted on a mobile object according to the present 
he steos of preparing a target apparatus for calibration around the mobile object; fix,ng a pos.t.on of the target appa- 
M lSrSri apparatus has a prescribed positional relation with the mobile object, by us.ng a posrt = g 
meansprevidedtoatSastoneof the mobile object and thetarget apparatus; and captunng a feature po.n of *etarg^ 
Apparatus with the camera, wherein the camera is calibrated based on a relation between .rnage coord.nates of the 
fpature Doint and real-world coordinates thereof. 

M A method for calibrating a camera mounted on a mobile object according to the present ,nven on ^dudes 
he steps of preparing a target apparatus for calibration around the mobile object; obta.n.ng a positional re at.cn be- 
tween the target apparatus and the mobile object by using a positional-relation estimating means proved to ^aUeast 
one of he mobile object and the target apparatus; and capturingafeaturepointofthetarge apparatus w.th he ^ 
wherein the camera is calibrated based on image coordinates of the feature point, real-worid coord.nates of he feature 
A based on the target apparatus, and the obtained positional relation between the target apparatus and the mob.le 
object. 

BRIEF DESCRIPTION OF THE DRAWINGS 

100211 Fias 1A and 1B are diagrams conceptually showing a technical idea of the present invention 

[0022] Figs. 2A and 2B are diagrams showing the structure of a calibration system according to a first embod.ment 

of the present invention. 

F00231 Fig 3 is a diagram showing another example of the structure of joint means. 

[0024] Fig. 4 is a diagram showing an example of the structure of a positional adjustment means. 

100251 Fig. 5 is a diagram showing an example of the structure of a scope means. 

0026 S 6 is a diagram showing another example of the structure of the calibration system according to the first 
embodiment of the present invention, in which scope means and positional adjustment means are provided. 
r00271 Fia 7 is a modification of the structure of Fig. 6. 

[0028] Figs. 8A and 8B are diagrams showing positional adjustment of the case where the scope means is prov.ded 

°o n 029r 9 Rg a s P oTand 9B are diagrams showing positional adjustment of the case where the scope means is provided 

°0 n 030 V ] ehi Rg. 10 is a block diagram functionally showing the structure of the calibration system according to the first 

STS! aTgram i«ng a calibration method, and showing the relation between a world coordinate 
system, a camera coordinate system and image coordinates based on a p.nhole camera. 

[0032] Fig. 12 is a diagram showing the structure of a calibration system accord.ng to a second embod.ment of the 

[0033] 1 ' RgflSA and 13B are diagrams showing positional adjustment of a target apparatus according to the second 
embodiment of the present invention. 
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[0034] Figs. 14A. 14B and 14C are diagrams illustrating the points to notice" regarding the mounting position of scope 
7S Fig. 15 is a block diagram functionally showing the structure of a calibration system according to a third em- 

fo r 03 e 7] nt ' Fig.Uis a diagram showing the structure of a calibration system according to a fourth embodiment of the 

Sq "Fig n, 18 "s a flowchart showing a flow of the process of calculating a coordinate value of a feature point. 
10 0039 Fig 19 is a diagram illustrating calculation of the coordinate value of the feature point 

[oS] Fig. 20 !s a diagram showing another structure of the calibration system according to the fourth embod.ment 

SS iC Rgs 22A 22B and 22C are diagrams illustrating the reason why precise camera calibration is required. 
[0043] Fig. 23 is a diagram showing the structure of a conventional vehicle monitonng system. 

DETAILED DESCRIPTION OF THE INVENTION 

[0044] Figs 1A and 1B are diagrams conceptually showing a technical idea of the present invention. In Fig. 1A, a 
Sfcto 2 having a camera 1 mounted thereon is positioned on the ground, and a target apparatus 3 for cahbrat.on ,s 
£d to the gTund ?n other words, the positional relation between the vehicle 2 and the target apparatus 3 ,s set by 

SoS ° Inttf Sse'of Fig 1 A, the vehicle 2 must be positioned on the ground with precision on the order of several 
miSters ^.0 taL Vpreciston calibration, bu, in practice, this is extremely difficult. Moreover, the veh.cle 
2 must be moved to the installation location of the target apparatus 3 upon every cahbrafion 
r0M61 Then in the present invention, the target apparatus 3 is directly fixed to the veh.cle 2 as shown ,n Rg. IB 
Sher than , b leans of the ground. As a result, the vehicle 2 need no longer be positioned with prec.s,on on ^e order 
oi 2veS miilimeL, nor is the location for conducting calibration limited, thereby a.iow.ng ,mplementat,on of s.mpte 

30 [OM^Hereinafter.embodimentsofthepresent^^ 
ings. 

(Embodiment 1) 

100481 Fias 2A and 2B are diagrams showing the structure of a calibration system according to the first embodiment 
oHhe present hwenflon In Figs. 2A and 2B, a vehicle 10 as a mobile object having cameras 101 mounted thereon 
anJ a tempera 30 for calibrating the cameras 101 are physically fixed to each other wKh jo.nt means ; 107 a 
b Sriioning means. Fig 2A shows the target apparatus 30 being joined from the rear of the veh.cle 
Mn^ 

107a at the target apparatus 30 are. e.g., threaded projections, and the jo.nt means 107b at the veh.cle 10 are, e.g., 

mm ^'target apparatus 30 has feature points PP that are required to adjust the cameras 101. In Figs. 2A and 
2B me featu e points PP are provided by the intersections of a lattice structure formed by rods of plastic, wood or 
^SSl £££ "feature points PP may be provided in any form, like being directly drawn on a p.ate-hke 

not specifically limited to the combination of threaded projection and tapped hole, and vanous sfructu ;^ are .P°^' e < ; 
F^amS a hole may be formed in advance in the vehicle 10 as a joint means so as to rece.ve a rod-l.kejo.nt means 

55 KrA^raK-mberasa^ 

he ta get apparatus 30 is fixed to the vehicle 10 by. e.g.. fitting the fixing member ,nto a port.cn of the veh.cle ,VU Fj. 
ahTdteaiwn showing an example of the structure using such fixing members. In the example of F.g. 3, p nch ng 
IZZ^Jl^m^™* ^e target apparatus 30 is physically fixed to the vehicle 10 by p.nch.ng the 
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TadvSnt - *• target apparatus 30. It should be appreciated that the fixmg member 

S adSment tftatal apparatus 30 with respect to the vehicle 10. The positional adjustment means 104 as 
^^ffS^.^Shaln 104a capable of being finely adjusted in the vertical direct™ wjh a screw and 
frilil 1Mb capable of being adjusted in position in the four directions (front, rear, nght and left) w.th a wheel 
Note TZ To de to prevent any positional displacement of the target apparatus 30 during calibrate the pos^nal 
adts i mean To^efeS has a fixing member for fixing the movement of the mechanisms 104a 104 b. How- 
adjustment ™ans iu« P re J> * standpoint of preventing the positional displacement of the target 

— " * *• " 101 can be carried ou : as r g as the 

1™ can maintain a constant positional relation between the vehicle 10 and the target apparatus 30. 
SiT Moreover scope means 105 as shown in Fig. 5 is used in the present embodiment. The scope means 105 
SL to see ffThe vehicle 10 and the target apparatus 30 have a prescribed positional relafion or not 

Z Tm lZZ of?*. S two v^ioally and horizontally graduated circular glasses 501 are mounted a. a presented 
ItenceZea^ 

Sr 9 Ftr r a a dSm showing an 

IJSUt fT 6 TTwSo^ oMhe target apparatus 30 can be easily adjusted by visually confirming target po.nts 
?06 on^^^^ especle scope means 105 of the target apparatus 30. The target pent 106 may 

iLtu Lr+^nnfthpvPhiriP rq a corner of the rear window, may be defined as the target point 106. The 

sCm^r^^ 

raosT' Note that as shown in Fig. 7, the scope means 105 may be provided on the vehicle 10 with the target point 
0 blin oSSon "e^et apparatus 30. The scope means 105 may be of either a fixed or detachable typa n 
he Se of Fig 7 nowever, ?he scope means 105 is desirably of a detachable type since rt ,s necessary only du ng 
SbTafion S nee the scope means 105 is provided in order to adjust the position of the target apparatus 30, .t .s 

tha *e Tcope means 105 is strongly fixed to the vehicle 10 or target apparatus 30 dunng cal.bratio^ 
S^^SSirSS be detachab.y mounted at a plurality of positions on the target apparatus 30. This 
Ses s^eTnTa mounting position of the scope means 105 according to 

cameras 101 on various vehicles 10 are to be calibrated. In this case, the posit.on of the target po.nt 106 on the vehicle 

■*—»» 30 is «** d whi,e iooking ,he target pow 106 thro , u9h the 

[0058] Pos ~^ app 7 ratus 3 o m ay be finely positioned with the positional adjustment means 

-me Je as that'eonnecting the view point and the centers of two 

fS' 9 F a iq S s eS 8Aa 1 nd 8B are diagrams showing the positional adjustment of the case where the scope means 1 1 05 is 
" ' 9 the taroet apparatus 30. Figs. 8A and 8B show the states before and after the posit,on.ng, respect.vely. 
provided on flnr target apparatus g positioning is set at the lower right end of the rear wmdow 

f ev^^^^^ 

on the same line as that connecting the view point and the centers of two crcular glasses 501. 
00601 F^gs 9A and 9 B are diagrams showing the positional adjustment of the case where the scope means 10 s 
1 J H.nLClhiriPio Fio QAandOBshowthestatesbeforeandafterthepositioning, respectively. Intheexample 
so S2 TlT!nd W ie S X 10 ^nS«oning is set at the centra, one of the feature points PP of the target 
30 and me posXn ofThe target apparatus 30 is finely adjusted such that the target point 106 is located on 

^ame^ 

m >SZ n pigs 8A and 8B or Figs. 9A and 9B, a target presenting means such as LED may be prov.ded at the 
taroet point 106 for improved visual recognition. Thus, the calibration efficiency can be .mproved. 
55 Xl Note that whether the scope means 105 is necessary or not is determined dependmg on a jo.nt method for 
KSLJ Sno L vehicle 10 and the target apparatus 30 to each other. More specifically, the scope means 105 s 
Set^ 

maintained as a result of fixing them. 
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ro0621 It is also possible to eliminate the need for the positional adjustment means 104 by providing the joint means 
hemselves with a function to finely adjust the position of the target apparatus 30. For example in the case where a 
rod-like member is used as the joint means 107a of the target apparatus 30 and a hole is used as the joint means 107b 
of the vehicle 10 so that the joint means 107a is inserted into the joint means 107b. the position of the target apparatus 
30 can be finely adjusted by varying the insertion depth of the joint means 1 07a. In this case, it is preferable to use the 
scope means 105 for positioning. It is also preferable that the positional relation between the vehicle 10 and the target 
apparatus 30 is fixed with a fixing member after the positional adjustment is completed. 

100631 Fiq 10 is a block diagram functionally showing the structure of the calibration system according to the present 
embodiment As shown in Fig. 10, the vehicle 10 includes, in addition to the cameras 101 , a camera calibration means 
108 for conducting c«librationofthe camera 

to the captured images of the cameras 101 for image conversion, and an image display means 103 such as a monitor 
fordisplaying the converted image. The vehicle 10 also includes the joint means 107b and target point 106 as described 
above The target apparatus 30 includes the positional adjustment means 104, scope means 105 and joint means 

r0u641 S Th^cSfera^nbration means 108. which conducts camera calibration based on calibration data described 
below is not an essential element to be provided within the vehicle 10, since a real-time processing is not always 
required. The camera calibration means 108 can be implemented with a general-purpose computer capable of oper- 
ating a calibration software. When implemented with a general-purpose computer, the camera calibration means 108 
may be placed at a remote location from the vehicle 10 so that the obtained calibration data may be transmitted to the 
computer by means of a storage medium such as floppy disk and memory card, or through commun.cat.on such as 
radio or cable communication. . 
[0065] Hereinafter, a camera calibration method will be described briefly with reference to Fig 11 For simplicity it 
is herein assumed that a pinhole camera is used in which lens distortion, displacement of the optical ax* and the like 
can be ignored. Fig. 11 is a diagram showing the relation between a world coordinate system, a camera coordinate 
system and image coordinates based on a pinhole camera model. In calibration, the position and onentation of the 
camera in the world coordinate system are calculated using the feature point PP. 

[00661 In Fig. 11 , f denotes the focal length of the camera. Pv(u. v) denotes an image coord.nate value of the feature 
point PP as captured with the camera, Pe(xe, Ye, Ze) denotes a coordinate value of the feature point PP .n the camera 
coordinate system. Pw(Xw. Yw. Zw) denotes a coordinate value of the feature point PP in the world coord.nate system. 
Provided that, regarding the relation between the camera coordinate system and the world coord.nate system, a 3 x 
3 rotation matrix for making the respective axes of the coordinate systems in parallel with each other is R(r1 1 , r33), 
and a translation vector for matching the respective origins is T(tx, ty. tz). Pv and Pw can be mapped w.th each other 
by the following equation: 
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[0067] Specifically, camera calibration is nothing less than a process of obtaining each element of the rotat.on matrix 
R (corresponding to the orientation of the camera) and each element of the translation vector T (corresponding to the 
position of the camera). In the above equation, the number of unknown parameters for determining the or.entat.on and 
position of the camera is six in total, i.e.. three for the rotating matrix R (there are nine matrix elements, but .ndependent 
parameters are only three rotation angles about the respective axes X, Y and Z because this is a rotation matnx). and 
three for the translation vector T. 

[00681 Two equations for u and v can be obtained by substituting a set of the .mage coordinate value Pv(u v) and 
ts corresponding coordinate value Pw(Xw, Yw, Zw) in the world coordinate system, Le a set of caKbrafion data .for 
the above equation. Accordingly, the aforementioned six parameter values can be obta.ned wrth at least three sets of 
calibration data. In practice, in order to reduce the effect of measurement error, the equation .s general* solved by the 
least-square method or the like using the calibration data regarding as many feature points as possible that are d.s- 
persed in a wide range. 
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r00691 In order to conduct the camera calibration with high precision, a multiplicity of points each having a known 
exact coordinate value in a predetermined coordinate system, i.e., feature points PP, must be prepared within the 
mentoring range of the camera. It is herein the most important to accurately set the feature points PP to respective 
potions with known coordinate values. This means that the target apparatus 30 having the feature points PP must 
be accurately positioned with respect to the vehicle 10. ^h^a 
[0070] For example, the calibration data is manually created by the following two steps using the accurately posrt l0 ned 
target apparatus 30: 

1 The feature point PP is captured with the camera, and a coordinate value Pv(u, v) of the feature point PP on 
the camera image is specified. The coordinate value is manually specified by, for example, taking the camera 
image into a computer for display on the screen, moving the mouse cursor to the position of the feature point on 
the screen and then clicking on that position with a mouse; and 

2 The coordinate value Pw(Xw, Yw, Zw) of the feature point PP in the world coordinate system is mapped with 
the specified coordinate value Pv(u, v) on the camera image, thereby creating the calibration data. 

[0071] As has been described above, according to the present embodiment, even an unskilled person can easily fix 
he positional relation between the mobile object and the target apparatus to a prescribed relation by the join means, 
scope means or the like. Accordingly, accurate positioning of the mobile object is no longer required for cal.braton 
thereby simplifying calibration of the cameras mounted on the mobile object. Moreover, calibration does not require a 
large space and therefore can be conducted at a small place. Moreover, the target apparatus can be realized in a 
portable, compact form. 

(Embodiment 2) 

[0072] According to the second embodiment of the present invention, the positional relation between the vehicle 10 
and the target apparatus 30 is fixed to a prescribed relation by using the scope means without making the vehicle 10 
and the target apparatus 30 in contact with each other. In other words, the joint means for physically fixing the vehicle 
10 and the target apparatus 30 as in the first embodiment is not used in the second embodiment. 
[0073] Fig 12 is a diagram showing the structure of a calibration system according to the present embodiment. In 
Fig 12, the vehicle 10 having the cameras 101 mounted thereon and the target apparatus 30 are not physically fixed 
to each other The target apparatus 30 is positioned by visually confirming the target points 106 on the vehicle 10 by 
the respective scope means 105 on the target apparatus 30. The positional adjustment means 104 arid the scope 
means 105 each has the same structure as that described in the first embodiment. The positional adjustment means 
104 and the scope means 105 form a positioning means according to the present embodiment. 
35 [0074] Figs. 13A and 13B are diagrams showing positional adjustment of the target apparatus 30 according to the 
present embodiment. Figs. 13A and 13B show the states before and after positioning, respectively In the examp e of 
Figs 13A and 13B, the target points 106 for positioning are respectively set at the lower nght and left corners of the 
rear window of the vehicle 10. The position of the target apparatus 30 is finely adjusted such that each target point 
106 is located on the same line as that connecting the view point and the centers of two crcular glasses 501 of the 

40 corresponding scope means 105. 

[0075] Note that the structure of the scope means 105 is not limited to that described herein, and another structure 
may also be possible. For example, a laser-beam emitting apparatus may be used as the scope means and the posrt.on 
of the target apparatus 30 may be adjusted such that the emitted laser beam light is incident on the target point 10* 
This is advantageous in that the position can be confirmed without looking through a scope. Alternately, a reflec «ng 
plate may be provided at the target point 106 in order to facilitate confirmation that the laser beam light is incident on 
the target point 106. The scope means may also be provided with a means for receiving the reflected light, so as to 
notify completion of the positional adjustment with a sound such as buzzer in response to recept.cn of the reflected 

liqht at the light-receiving means. 

[0076] Hereinafter, the points to notice will be described in terms of the mounting positions of the scope means. 
Depending on the mounting position of the scope means, there may possibly be many ^"^t^*^? 
where the target point and the centers of the two circular glasses exactly match each other. This wiH be described with 
reference to Fig. 14A, 14B and 14C. In Figs. 14A, 14B and 14C. it is assumed that positioning of the target apparatus 
is conducted using two scope means, and that the target apparatus 30 has a fixed height and is movable in the four 
directions (front, rear, right and left) on the plane where the vehicle 10 is located (i.e., the road surface S). 
[0077] In the case of Fig. 14A. a line of sight VL1 through the scope means 105 is in parallel with the road surface 
S, and extends through the target point 106. In this case, there are many positions of the target apparatus 30 where 
the respective target point 106 and the centers of the two circular glasses 501 of each scope means 105 provided on 
the target means 30 match each other. For example, as shown in Fig. 14B, when the target points 106 at the upper 
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riaht and left corners of the rear window of the vehicle 10 are viewed from'each of the positions VP-1. VP-2 VP-3. 
efch target point 106 exactly matches the centers of the two circular glasses 501 when v.ewed through the scope 
mMn5 105 This makes it difficult to locate the target apparatus 30 at a correct position. 

m07ffl For exa^tn order to avoid such a disadvantage, a line of sight VL2 through the scope means 105 may 
b rendered at an angle with the plane on which the target apparatus 30 is moved, i.e., the road surface S. as shown 
n in the case'of using three or more scope means as well, the positions of the scope means and 

the target points must be carefully determined in view of the possibility of the same d.sadvantage. 

(Embodiment 3) 

[0079] Calibration data may be manually created in the case of using only a small number of feature points or cali- 
brattna onC small number of cameras. However, if a large number of feature points are used or a large number of 
cameras must be calibrated, such a manual operation would be extremely time-consuming in terms of cal.brat on. 
MoTeTver^ 

r h Trd for the Aerator and also the tired operator may wrongly input the coordinates. wh.ch may result in degraded 

r n 008m On Fiq 15 is a block diagram functionally showing the structure of a calibration system according to the third 
ImZment of the present invention. In Fig. 15. the same components as those of 10 are denoted w,t , the same 
reTerenrnumerals and characters, and detailed description thereof is herein omitted. The veh.c b tt iMM 
meansToo for controlling the target apparatus 30. and storage means for storing control data 11 1 1 for automatically 
TaSng The caSon data. The target apparatus 30 includes feature-point generating means 110 for generating a 
fpature Doint accordinq to a control signal from the control means 109. 

00811 The cont ol means 109 transmits a control signal to the feature-point generating means 110 so as to cause 
!e featun3-Dorqenerating means 110 to generate a feature point required to adjust the camera 101. whereby the 
SbraTon dSs 9 aufomi,y or semi-automatically created. Thus, high-speed, simplified calibration of the camera 
101 is imolemented. Specific processing will be described later. «„ 

0082] Fig 16 is a diagram showing the structure of a calibration system according to the present embedment n 
Fig ?6 the feature-poinlgenerating means 110 generates a feature point by using light-em.tt.ng means 110a More 
soec fioalv an internal driving means (not shown) receives a control signal from the control means 109 and causes 

heS^ 

is a nllaTcCD (charge coupled device) camera or an infrared camera, an electron* flash, incandescent lamp or the 

2? F^xaCe an iSe^Hfarence method is effective as a method for automatically detecting a Hgh, 
SSg ^ZTlZtoLve difference method, the difference between successive two frames ,s calculated -n 
S extract the position having the maximum difference. In the case of using the mter-frame drfference method, 
the process of extracting a feature point includes the following steps: 

/<5yn ralrulation of the difference between frames is initiated; 

S the cS means 109 transmits to the target apparatus 30 a control signal for initiating l,ght-em,ss.on of a 

rSTS^heleature-point generating means 110 receives the control signal, and causes light emission of the corre- 

SSIftSK exTacted that has the maximum integral value of the difference between frames during a pre- 
^hed time Deriod after the control means 109 outputs the control signal; and 

(sTsfreSng toe frame extracted in Step ST4, a coordinate value of the point having the peak difference is 
obLined ? and this coordinate value mapped with a spatial coordinate value of the light-em.tt.ng feature po.nt ,s 
stored as calibration data of the camera 101. 

[0085] By conducting the series of Steps ST1 to ST5 for every controllable feature point, the calibration data of the 
Snera 101 Te the data having spatial coordinates of each feature point mapped with a coord.nate value on the 
^uSd laaecan teautomaticallyprodu^d.Thus, efficiency of the camera calibration canbes.gn,f.cantly.m^ 
S^^StlSS^Soo-i as described above, the control means 109 retains in advance the data 
S the numbed' erteature p'oint of the target apparatus 30 mapped with spatial coordinates thereof as the 
control data 111. Table 1 below is an example of the control data 111. 
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10 



15 



20 



Feature Point No. 


Spatial Coordinate Value of Feature Point 


1 


(X1.Y1.Z1) 


2 


(X2, Y2, 72) 






N 


(XN, YN, ZN) 



111 for such 


a case. 










[Table 2] 


25 


Installation Position No. 


Feature Point No. 


Spatial Coordinate Value of Feature Point 




1 


1 


(X11.Y11.Z11) 




1 


2 


(X12, Y12, Z12) 



30 



55 



1 


N 


(X1N, Y1N.Z1N) 


2 


1 


(X21, Y21,Z21) 


2 


2 


(X22, Y22, Z22) 








2 


N 


(X2N, Y2N, Z2N) 


M 


1 


r (XM1.YM1.ZM1) 


M 


2 


(XM2, YM2, ZM2) 








M 


N 


(XMN, YMN, ZMN) 



poM matters and nrairtk a patrol alga* to M <™*£^£J°Z£Z£&™*n. coorttah. 
poU co^apondiag to «. obtoined M» .1. k ".appad with , 
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otm. a!Z*n\ has occunrt or not can be easily ootermlned b, installing the target apparatus 30 and raoa- 

has been described above, according to the present embodiment, collection of calibration data, which is 
Snsome in tho : manual operation, can be conducted automatica.lv, whereby the calibrate efficency can be s,g- 
nificantly improved. 

(Embodiment 4) 

he cond^on «5 a" l" the target points 106 required for calculation are included in the stereo camera , .mages. However 
%%S2£Z ! n-» to set based on an, point of the ta-set ap^abra 30. Not. that a camera parameter of 
ISTESL,* in Fig. 19, provided ,hat tto baeoline ot tto s.areo cama„s Is b. .to ooordlneto veto* o, tto 
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the principles of triangulation: 

X1 = b(XI + Xr)/2d; 



Y1 = b(YI + Yr)/2d; 



and 



Z1 = bf/d. 

The respective coordinate values of the target point 106 in the right and left camera images, (Xr. Yr) and (XI. Yl), may 

STTf JSS SSSSj thJ ^n^Tese Li parameters are given by the following 
equation: 



y 2 

\ Z 2/ 



f 



•(2) 



mm* m Enuation (2) ri is an element of a 3 x 3 matrix representing rotation for making the coordinate axes of the 
^es^fe^ 

iTcalSated ^More s eXS s^ce the parameters representing the positional relation between the coordinate 
l^Z 1 and 2 are obtained in Step S3, the coordinate value of each feature point in the coordinate system 1 can 

the ore-measured coordinate value of the feature point in the coordinate system 1 as (XI. Y1. Z1) of Equation (2). 
01051 Then Tn Step 86. the coordinate values of the feature points in the coordinate system 2 are transferred to the 
[0105] i nenjn wepjs ,o, exam ple, this coordinate value data may be in such a form as 

SITiS^JTSl^^^ ^ -ordinate system 2 of the feature points caught by the 

S£T No hat, in the present embodiment, the stereo cameras are used to measure the coordinate values n the 
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10 may sequentially cause the feature points of the target apparatus 30 to emit light. 
(Applications to Services) 

f0108l The effects of the present invention allow the camera calibration to be developed as service businesses at 

SSsjSsssssBssss: 

ST Aiming that the calibration according to the present invention will be actually carried out at service centers 
in the future, the procedures thereof will be briefly described with reference to Fig. 21 . 

' Thelarget apparatus 30 stored at the service center is installed near the vehicle 10 having the cameras 101 
to bJSlSed S thereon. The installation position of the target apparatus 30 «s determined such that the 
SrgettroS means 114 of the target apparatus 30 can catch the targe, points 106 on the vehicle 10 and 
She cameras 101 catch the feature points on the target apparatus 30 as many as possible. 

" ^helarget points 106 are caugM^ 

stereo cameras are used as the target data obtaining means 114, images including the target panto 106 are 

captured with both the right and left cameras. 

' ^positional relation between the vehicle 10 and the target apparatus 30 » <"^l££*^ 
to*ino moan* 113 and the coordinate values of the feature points as viewed from the vehicle 10 are calculates 

or by means of a storage medium such as floppy disk and memory card. 
" ^ ^camera calibration means 108 calibrates each camera 101 based on both the image coordinates of the 
featura poTntoSptreTvl the cameras 101 a 

aPP Sfth»t in the case where the respective positions and orientations of the cameras 101 are different and 
not aTo^am^ cl be Vibrated by conducting the series of Steps SS1 to SS4 only once, the series 

roar *irfp of the vehicle but also on the front, right and left sides thereof. 

SelTin ea* of me above embodiments, the image converting means 102 and the display means 103 
are %^Ji£^*** «• However, these means may alternatively be provided at 

Zeove! the present invention is also applicable in the case where the cameras are mounted on the mobile 
objects other than the vehicles, such as trains, airplanes or robots. 

(Supplementary Explanation) 

rnum in the case where the cameras are merely used for monitoring, not so precise calibration is required. For 
effective in such applications. 

rm 1 ii ThP reason whv such precise calibration is required will be supplemented. 

I s Z camera 101 monitoring the road surface rearward is 
as shown in Fig. 22A. As shown in Fig. 22B, the camera 101 is mounted at a he ig ht h of 1,000 mm above the ground, 

^T^Z^^^ of the camera 101 is displaced and the orientation 8 thereof is shifted 
HnwnLd bvT(-T) as shown in Fig. 22B. At this time, the point on the road surface corresponding to the center 
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•» * 

P1 and P2 is given by the following equation: 

dl = h x (tan0 - tan(8 - d0)) 
= 1,000 x (tan60° - tan59°) 
= 67.8 mm. 



10 



15 



20 



19 cn. k P-odueed acting » I. TT^^^^SJI-ln. larger than 0. the wltne 

Image, the white Ikte k net rrtMHts on the CCD als0 Iesu „ 5 ,„ law positional dis- 

,0,141 Moroov.r. " "f^^otS 47m^« 4.85 mm) end , wide-engle lens M . 



25 



30 



35 



dp = atan(du/fl) 
= atan( 0.1/3) 

= 1.91°. 



40 



45 



Therefore poin^^ 

101 15] Thus, displacement of several degrees in «M J ^ 0 ^ or more on the road 

positional relation between the CCD and the te ^"^ the lens and the CCD 

surface. Accordingly, regarding the onentation of the ' h ' P °~ ^, t0 or sma ller than these values, 
within the camera, calibration must be ^^^^^^^^ relation between the mobile 

Claims 

1 . A calibration system, comprising: 

a mobile object having a camera mounted thereon; 

relation between the mobile object and the target apparatus to a prescnbed relahon. 
The calibration system according to c.aim 1, wherein the positioningmeans has a joint means for physical* fixing 
the target apparatus to the mobile object. 



50 2. 



55 located on the other. 

4. 
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feature point of the target apparatus. 
5. A calibration system, comprising: 

a mobile object having a camera mounted thereon; 

for obtaining a positional relation between the mobile object and the target apparatus. 

Amethad lor M a camera mounted on a mobile .Died, comprtsing the steps of 
prar^n».terg.lapparatesl«<»l«^^ 

coordinates thereof. 

10 A method for calibrating a camera mounted on a mobile object, comprising the steps of. 
the mobile object. 
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FIG. 2A 
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FIG. 8A 
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FIG. 9A 
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FIG. 11 
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Coordinates in Coordinate System 1 : (XI, Yl, Zl) 
Coordinates in Coordinate System 2:(X2, Y2, Z2) 
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FIG. 22A 
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FIG. 23 
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